METHOD AND APPARATUS FOR ACTIVE GRAPHICAL CONTROLS 



FIELD OF THE INVENTION 

This invention relates generally to graphical controls and more specifically to active 
graphical controls that maintain a predetermined level of contrast against a background. 

BACKGROUND OF THE INVENTION 
Desktop computers, portable computers, PDA's, digital cameras, handheld 
game machines and any device that has a display diat can show a background scene can 
have a problem when the graphical control for the user interface is close to the color of the 
background. For example a digital camera typically uses the onboard display to show the 
user the digital images recorded by the camera. The display also typically has some user 
mterface controls shown to allow the user to navigate among the unages stored in the 
camera. When the color in the image being displayed closely matches the color used to 
display the graphical control of the user interface, the graphical control may become 
difficult to see. Another example is the desktop of the Microsoft Windows ™ operating 
system. When a background or pattern is displayed on the desktop that closely matches the 
color of one of the icons or programs on the desktop, the icon or program may become 
difficult to see. Therefore there is a need for a graphical control or user interface that 
maintams a predetermined contrast against the area where the graphical control is being 
displayed. 



SUMMARY OF THE INVENTION 
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An active graphical control that automatically maintains a minimum level of contrast 
between the control and the background against which the control is being displayed. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a drawing of a display with graphical controls that have a similar color to 
the background that is displayed. 

Figure 2 is a flow chart of one embodiment of the present invention. 
Figure 3 is a drawing of a close up of a segment of a graphical control. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

Some digital devices have two levels of programs running on the device and some 
have only one level of programs running on the device. Portable computers typically have 
two levels of programs running. An operating system is one level of programming and then 
the programs running on the operating system are the second level. An example of a 
program that would run on the operating system is a word processing program. Operating 
systems are programs that typically control, or provide an interface for, the hardware of the 
device. For example, operating systems typically control the allocation of memory, the 
access to storage, the access to the display device, and the input/output channels of the 
device. Digital cameras typically only have one level of program operating on the device, 
typically called firmware. The firmware on a digital camera typically acts like an operatmg 
system for the device. Firmware for today's digital cameras typically don't provide an 
environment for other programs to operate. However in the future digital cameras may 
include operating systems that do allow a second level of programs to operate. 
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A device that only has one level of programs running on it typically has total control 
over what is currently shown on the display. The device knows the location and color of all 
graphical controls and can determine the color of the pixels near or adjacent to the 
graphical controls. This can be done by evaluating the copy of the background displayed or 
by reading the display memory directly. Or there may be special hardware in the device 
that allows this operation to be done in hardware. For example the device may contain a 
graphical processor that has a background display plane and a foreground display overlay. 
The graphical processor may be configured to automatically maintain a predetermined level 
of contrast between elements in the foreground display overlay compared to the current 
content of the background display plane. 

The operating system for a device that has more than one level of programs may 
show controls on the display or it may allocate portions of the display to a second level 
program running on top of the operating system. In this case the operating system may not 
directly control what is currently shown on the display. The operating system may not 
control what is displayed by the second level program, but the operating system can 
typically determine what is being displayed on the display device. The first level program 
(operating system) may use its own system level calls to determine the content of the 
display or it may use lower level commands directly. The lower level command may access 
the video memory or may read a copy of the files being displayed. 

A second level program that is using the display may control all or a portion of the 
display. The second level program typically will know what is being displayed on the 
portions of the display device that it controls. Using common system calls the second level 
program can find out what is being shown on the display near or next to the area of the 
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display that it controls. For example under the Microsoft Windows ™ operating system, a 
program could use GetDiBitsQ to determine the current content of an area of the display. 
Therefore all three types of programs (firmware, operating systems, and second level 
programs) can determine what is currently being shown on the display. 

The display area of a device can be used for many purposes. Typically the display 
area is used to show controls for programs or controls for the user interface for the device. 
In many cases the program controls or user interface controls are shown on the display 
device against a background (see figure 1). A Background can be an image, a pattern, a 
color, a video or the like. Most operating systems allow the user to select what is displayed 
for the background. Some devices, for example digital cameras, display images on the 
device and have the graphical controls for the camera's user interface superunposed on top 
of the image. In this case the image is considered the background. When the background 
102 is close to a color on the graphical control 104 for a program or device, the graphical 
control 104 may become difficult to see. A graphical control may be an icon, a bit map, 
text, or the like, shown on the display to enable user interaction with the program or 
device. For example a graphical control could be, but is not limited to, a drop down menu, 
a button, an arrow, crosshairs, edit boxes, icons, drawings, thumbnails, or pictures. 

In one embodiment of the current invention a digital device would have a display on 
the device. The device would have the ability to show backgrounds on the display (see 
figure 2). The device would also show active graphical controls superimposed on the 
background 202. An active graphical control is a control that maintains a mmimum level of 
contrast between the graphical control and the background against which the graphical 
control is shown. The level of contrast between the graphical control and the background is 
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compared to a threshold 204. When the level of contrast is below the threshold the visual 
appearance is changed 206 by either changing the visual appearance of the graphical control 
or by changing the visual appearance of the background near where the graphical control is 
shown or by changing both the background and the graphical control. One way of changing 
the visual appearance is to change the color. In this application changing the color includes 
changing the hue, the saturation, the intensity, or a combination of the hue, saturation, and 
uitensity. 

The display devices used today typically are made up from small elements called 
pixels. One way to measure contrast is to compare the color of a background pixel with the 
color of a pixel on the graphical control that is adjacent to or near the background pixel (see 
figure 3). For ease of understanding a reduced number of pixels has be depicted in figure 3. 
Some pixels on graphical controls may have more than one background pixel that is 
adjacent to the graphical control pixel. For example pixel 302 on graphical control 304 is 
adjacent to background pixels 305, 308, and 310. In one embodunent of the current 
invention the contrast between each background pixel that is adjacent to a graphical control 
pixel is calculated. In another embodiment the contrast between the average of the adjacent 
background pixels and the graphical control pixel is calculated. For example the average of 
background pixels 312, 314, and 316 would be compared to graphical control pixel 318 and 
a second average of background pixels 320, 322 and 324 would also be compared to 
graphical control pixel 318. In another embodiment the contrast between a background 
pixel and all graphical control pixels that are adjacent to the background pixel is calculated. 
In another embodiment the contrast is calculated between background pixels and graphical 
control pixels that have the most connecting surfaces. For example a graphical control pixel 
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318 with a common edge with one background pixel 314 and only a common comer with 
other background pixels 316 and 312 would be compared with the background pixel 314 
having the common edge. Typically in today's high resolution displays graphical controls 
are thicker than one pixel. The outer perimeter of a graphical control is defined by the 
pixels around the edge of the graphical control. The contrast between background pixels 
and graphical control pixels does not need to be limited to pixels that are adjacent to each 
other. Areas or regions of the background or graphical control can also be used to calculate 
a contrast measure. Some graphical controls contain pixels that are considered transparent, 
for example icons in the Microsoft Windows ™ environment. These pixels are part of the 
graphical control but allow the background to show through to the user. In graphical 
controls that contain transparent pixels the contrast may be calculated between the 
background pixels showing through the transparent pixels and the non-transparent pixels in 
the graphical control. 

A graphical control may be made from multiple elements. For example when the 
graphical control is text, each letter in the text string may be a graphical element. When the 
graphical control has multiple elements each element may be checked for proper contrast. 
When the contrast for a single element and that elements background falls below a 
threshold, the visual appearance of just that element or background may be changed or the 
visual appearance of all the elements may be changed. 

There are many ways to measure contrast. One way is to use a measure of the 
smallest noticeable difference or AL. The AL can be calculated using the red, green, and 
blue pixel values of the background and graphical control pixels. If the difference in color 
between the backgroimd pixel and the graphical control pixel is smaller dian a 
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predetermined number of AL's then the visual appearance would be changed. AL 
calculations are well known in the arts. Another way to calculate a contrast is to compare 
the difference between the each color component (R, G, B) in the background to the 
corresponding color component in the graphical control. For example compare the 
difference between the red background pixel and the red graphical control pixel with a 
threshold level (is Rl - R2 < T). When all color components (R, G, and B) have a 
difference below the threshold, the visual appearance would be changed. Another way, 
similar to the AL calculation, is to use a predetermined color amount. Using this method 
the RGB components of the background pixel and the graphical control pixel would be 
converted into a color space and then the distance between the two pixels would be 
calculated. If the distance is below a predetermined amount the visual appearance would be 
changed. There are many different color spaces that may be used, CIE, Munsell, and LAB 
color spaces are a few examples. 

When the contrast between the background and the graphical control has been 
determined to be below the desired amount the visual appearance will be changed. The 
visual appearance of the background, the graphical control or a combination of both can be 
changed. Changing the visual appearance can be done on an individual pixel level or can be 
done on a macro level. In the first case only the pixels that have contrast below the 
threshold would be changed and in the second case the visual appearance of pixels who's 
contrast is already above the threshold may be changed. For example given a graphical 
control having a color, and a background that has multiple colors, and some of the 
background colors causes the contrast level between the background and the graphical 
control to fall below the predetermined level, you have a number of choices on what to 

H.P. docket Number 10014022 page 7 Of 14 



change. In one embodiment only the color of the background pixels who's contrast falls 
below the threshold would be changed. In another embodiment the color of the pixels in the 
graphical control who's contrast falls below the threshold would be changed. In another 
embodiment both the graphical control and the background pixels would be changed when 
the contrast falls below the threshold. In another embodiment the color of all the pixels in 
the graphical control would be changed. When all the pixels in the graphical control are 
changed the new color selected would be chosen such that all pixels in the graphical control 
would have a contrast higher than the threshold. When changing the color of a pixel the 
hue, saturation, intensity or a combination of any two or a combination of all three can be 
changed. 

In a device that only has one level of program running on the device, the program 
would cause the change in visual appearance. In a device having more than one level of 
programs operating on it, the first level program may change the visual appearance or the 
higher level programs may change the visual appearance. 

The foregoing description of the present invention has been presented for purposes 
of illustration and description. It is not intended to be exhaustive or to limit the invention to 
the precise form disclosed, and other modifications and variations may be possible in light 
of the above teachings. The embodiment was chosen and described in order to best explain 
the principles of the invention and its practical application to thereby enable others skilled 
in the art to best utilize the invention in various embodiments and various modifications as 
are suited to the particular use contemplated. It is intended that the appended claims be 
construed to include other alternative embodiments of the invention except insofar as 
limited by the prior art. 
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